Likelihood Ratio Gradient Estimationfor Stochastic
نویسندگان
چکیده
In this paper, we develop mathematical machinery for verifying that a broad class of general state space Markov chains reacts smoothly to certain types of perturbations in the underlying transition structure. Our main result provides conditions under which the stationary probability measure of an ergodic Harris recurrent Markov chain is diierentiable in a certain strong sense. The approach is based on likelihood ratio \change-of-measure" arguments, and leads directly to a \likelihood ratio gradient estimator" that can be computed numerically.
منابع مشابه
Stochastic Gradient Estimation
We consider the problem of efficiently estimating gradients from stochastic simulation. Although the primary motivation is their use in simulation optimization, the resulting estimators can also be useful in other ways, e.g., sensitivity analysis. The main approaches described are finite differences (including simultaneous perturbations), perturbation analysis, the likelihood ratio/score functi...
متن کاملAd - a 263 766 Iii : R Copy - Keep This Copy for Reproduction Purposes
This paper is concerned with gradient estimation techniques for steady-state performance measures associated with regenerative stochastic processes. The principal emphasis is on the discussion of conditions under which likelihood ratio methods and infinitesimal perturbation analysis techniques are valid.
متن کاملEvaluating the Variance of Likelihood-Ratio Gradient Estimators
The likelihood-ratio method is often used to estimate gradients of stochastic computations, for which baselines are required to reduce the estimation variance. Many types of baselines have been proposed, although their degree of optimality is not well understood. In this study, we establish a novel framework of gradient estimation that includes most of the common gradient estimators as special ...
متن کاملRobust Analysis in Stochastic Simulation: Computation and Performance Guarantees
Any performance analysis based on stochastic simulation is subject to the errors inherent in misspecifying the modeling assumptions, particularly the input distributions. In situations with little support from data, we investigate the use of worst-case analysis to analyze these errors, by representing the partial, nonparametric knowledge of the input models via optimization constraints. We stud...
متن کاملConjugate gradient neural network in prediction of clay behavior and parameters sensitivities
The use of artificial neural networks has increased in many areas of engineering. In particular, this method has been applied to many geotechnical engineering problems and demonstrated some degree of success. A review of the literature reveals that it has been used successfully in modeling soil behavior, site characterization, earth retaining structures, settlement of structures, slope stabilit...
متن کامل